This invention generally relates to testing computer applications, and more specifically, to selecting test cases for such testing.
When a software application is upgraded, the application may be tested to determine if the upgrade introduces errors and/or bugs in the application. These new errors are known as regressions, and thus testing for the errors is referred to as regression testing.
This testing may include the design, development, and execution of a number of test cases. Each test case generally targets a specific element of functionality or individual interface of a software module, and includes a number of test conditions, a test script to be executed to test the conditions, and the expected result for each major step in the script.
The use of automated testing provides for more complete testing of software modules as well as more efficient regression testing of the software as changes to the modules are implemented.
Various test cases utilized in regression testing may be collected into test suites. For example, one test suite may be created to include all test scripts for a specific software module, and another created for a particular element of functionality across all modules.
Thousands of test cases may be developed for an application. Executing the entire system of test cases developed during software system testing can become expensive and time consuming. Accordingly, often not all the test cases are selected for the testing. Typically, for a large and complex system comprising thousands of test cases, the software test engineers intuitively select regression tests based on their experience and knowledge on program change specifications that need to be re-executed.
As the software development methodologies evolve into more dynamic and iterative ways of delivering code, the need for effective testing has become more and more critical. It is well known that automating a set of regression test cases is an efficient bug catching technique that allows testers to spot new defects introduced by new code in functional and non-functional areas (enhancements, fixes, configuration changes, etc.) and as well as reuse the automated scripts created for this testing purpose in different upcoming releases.
Test managers constantly circumvent the “Coverage vs. Time & Available Resources” constraints in their projects as it is not feasible to have 100% test coverage and neither automate and maintain up to date a huge number of test scripts.
As thousands of lines of code are persistently being introduced to the different modules of programs, the test team needs to effectively identify what test cases are best candidates for regression test phase and which ones of that regression universe are automation worthy.